Database context - Put the database back in the Query and Invoke script methods - #10579
Database context - Put the database back in the Query and Invoke script methods#10579andreasjordan wants to merge 2 commits into
Conversation
|
gonna put this through a round of ChatGPT Pro since it's such a sensitive change |
|
Glad I checked! FindingsP1: The restore check fails on case-sensitive SQL Server instancesFile: All four methods use this comparison: $connectionContext.CurrentDatabase -ne $previousDatabasePowerShell string comparison operators are case-insensitive unless the At minimum, use if ($previousDatabase -and $connectionContext.CurrentDatabase -cne $previousDatabase) {An ordinal comparison is more explicit: $databaseChanged = -not [string]::Equals(
$connectionContext.CurrentDatabase,
$previousDatabase,
[System.StringComparison]::Ordinal
)A server-collation-aware comparison would be exact, but an ordinal comparison is safe here. At worst it performs an unnecessary restore; it does not miss a real database change. This also needs a regression test using database names that differ only by case on a case-sensitive instance. P2: A restoration failure can hide the real error, or make a successful command appear to have failedFile: The restoration command is unguarded inside each $null = $connectionContext.ExecuteNonQuery("USE [$escapedDatabase]")There are two problematic outcomes:
The wrapper should retain the original P2: The temporary-table test does not prove that the wrapper uses the same sessionFile: The test currently:
A copied-connection implementation would also pass this test. The temporary table remains on the original connection regardless of which session executed Query the temporary table through the wrapper: $null = $callerServer.ConnectionContext.ExecuteNonQuery(
"CREATE TABLE #dbatoolsci_marker (id INT)"
)
$result = $callerServer.Databases[$contextDbName].Query(
"SELECT OBJECT_ID('tempdb..#dbatoolsci_marker') AS object_id"
)
$result.object_id | Should -Not -BeNullOrEmptyOr compare $callerSpid = $callerServer.ConnectionContext.ExecuteScalar("SELECT @@SPID")
$querySpid = $callerServer.Databases[$contextDbName].Query(
"SELECT @@SPID AS spid"
).spid
$querySpid | Should -Be $callerSpidVerdictRequest changes. Restoring the original context on the existing session is the correct overall approach, and the database-name escaping is correct. However, the case-insensitive comparison leaves the original bug unfixed on case-sensitive instances. The cleanup error handling also introduces ambiguous and potentially dangerous failure reporting. The temporary-table test should be corrected so it actually locks in the same-session guarantee. |
…pt methods The Query and Invoke script methods of Server and Database do not run on a private connection. The execution manager of an SMO database is the connection context of the parent server, which belongs to the caller, so these methods issued a USE and never switched back. Every command using them handed the connection back pointing at a different database, and everything the caller ran afterwards silently executed in the wrong one. All four methods now remember ConnectionContext.CurrentDatabase and put it back in a finally, so a failing query restores it too. The Server pair needs the same treatment of its own, because Server.Query and Server.Invoke call $this.Databases[$Database].ExecuteWithResults() directly and never go through the Database methods. Restoring rather than running on a copied connection is deliberate. A copy works, but it is a different session: it cannot see the temp tables or SET options of the caller, and it opens a connection per call. Restoring keeps the session, and costs one round trip only when the database actually moved. The database the caller was on is restored, not master. A connection sitting in msdb is returned to msdb - restoring to master would have passed every other test and still been wrong. This covers the script methods only. The direct SMO calls of #10555, and SMO's own Create() and Drop(), are untouched and still leak - Invoke-DbaDbUpgrade in #10556 is one of those. (do *) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… become the outcome Two fixes to the four script methods, both from the review on #10579. The comparison that decides whether the database has to be put back was case insensitive, so on an instance with a case sensitive collation it reported AppDb and appdb as equal and skipped the restore, leaving the caller in the wrong database - the very leak this change is about, on a valid configuration. It is -cne now, which cannot restore needlessly, because both sides are read from the same property and one database always spells itself the same way. The USE in the finally was unguarded, so a statement that made the previous database unreachable - taking it offline, dropping it, renaming it, revoking access - threw although it had succeeded, and a caller reading that as "it did not run" might run it a second time. A failing statement had its own error replaced for the same reason. Restoring is housekeeping and warns now instead of throwing. A Write-Warning inside a script method can be suppressed by the caller through WarningPreference or WarningAction, but it cannot be captured with WarningVariable or 3>&1, so the test sets the preference and asserts on behaviour rather than on the warning text. Tests: the temporary table test proved nothing about the session, because the table never left the caller's connection and a copied context would have passed it as well. It queries the marker through the wrapper and compares @@spid now, so the candidate that was rejected in the design fails it. Two new contexts. The failing restore returns normally and really does take the database offline. Databases whose names differ only in case are told apart - guarded by a BeforeDiscovery probe of the instance collation, because the scenario cannot be built at all on a case insensitive instance. It skips on the current CI instance and passes against a case sensitive one. Both were verified to fail against the old code. 16 test files of wrapper using commands, 116 tests, no failures. (do Connect-DbaInstance, Invoke-DbaQuery, Get-DbaDatabase)
5f89bbb to
cfce992
Compare
|
All three findings were right and all three are fixed. Two of them I could reproduce, and one of those needed a lab that did not exist yet. P1 — the case insensitive comparisonCorrect, and it is the one that needed new hardware. Three case sensitive instances now exist in the lab ( All four methods compare with The caller was left in the other database, which is exactly the leak this PR removes, on a valid configuration. Where the regression test runs. Its P2 — the restore could replace the outcomeAlso correct, and it is not theoretical. Reproduced with a caller sitting in its own database and a statement that takes that database offline: The I did not take the suggestion of throwing a distinct error when the statement succeeded but the restore failed. Throwing on a statement that already ran is the dangerous half of the finding — the caller most likely to be hurt by it is one that retries — and a distinct exception type does not help a caller that simply propagates it. Restoring is housekeeping and should not be able to decide the outcome of the call. Worth recording for anyone testing this: a P3 — the temporary table test proved nothingRight, and for the reason given: the table never left the caller's session, so a copied connection would have passed too. It now queries the marker through the wrapper and compares TestsVerified to have teeth by reverting the source and keeping the tests — both new cases fail against the old code with the diagnostics above.
One small discovery from building the fixture: two databases whose names differ only in case need distinct file names, because NTFS is case insensitive and the derived This text was created by Claude and reviewed by Andreas Jordan. |
Ten call sites went through a database object only because they needed somewhere to run a statement. The execution manager of an SMO database is the connection context of the parent server, so each of them issued a USE and left the connection of the caller in master or msdb. None of the statements needed a database context in the first place. They now run on the connection itself: - Export-DbaLogin, New-DbaLogin, Get-LoginPasswordHash read a password hash from sys.sql_logins or sys.server_principals. In all three the primary path already used ConnectionContext.ExecuteScalar and only the fallback went through master. - Get-DbaDbDetachedFileInfo resolves a collation with fn_helpcollations, which is available in every database. - Get-OfflineSqlFileStructure reads SERVERPROPERTY. - Set-DbaTempDbConfig executes ALTER DATABASE tempdb statements. - Remove-DbaAgentJob called sp_delete_job in msdb. The procedure is now named in full as msdb.dbo.sp_delete_job, so the connection does not have to go there. The help of Connect-DbaInstance recommended the pattern this removes, so it now points at the connection context and says why. This is the part of #10555 that needs no new mechanism, so it is separate from the script method fix in #10579. Set-DbaTempDbConfig also reads tempdb through $server.Databases['tempdb'].Query(), which is that other fix; the command is only free of the leak once both are in. (do Export-DbaLogin, New-DbaLogin, Get-DbaDbDetachedFileInfo, Set-DbaTempDbConfig, Remove-DbaAgentJob, Sync-DbaLoginPassword, Connect-DbaInstance) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…wo tests teeth Three fixes from the review on #10580. Set-DbaTempDbConfig cannot avoid moving the database context the way the other sites in #10555 could. FILEPROPERTY only reports on the current database, and the batches built for -Force carry an explicit USE [tempdb] because DBCC SHRINKFILE empties a file of the current database only. So the command remembers the database of the caller and puts it back at each of the two places that move it, the second in a finally, because a reconfiguration that fails half way moves it just as much. Measured against a connection that starts in msdb: it ended in tempdb before, it ends in msdb now, and that holds on the failure path too, which is the one a real forced reduction is most likely to take. The two file path lookups did not need tempdb at all and now read sys.master_files on the connection, which is a server level view. The comparison is case sensitive, for the reason given in #10579. sys.dm_db_file_space_usage was considered for the used space of the data files, which would have made the whole command server level, and rejected: its allocated_extent_page_count does not agree with FILEPROPERTY SpaceUsed - 0.50 against 0.56 MB on the same file - and a leak fix is no place to change what a size check measures. Tests: the forced reduction that the file already performs now runs through a connection that starts in msdb, so it asserts the database of the caller is left alone without reconfiguring tempdb a second time. The collation assertion in Get-DbaDbDetachedFileInfo compared against nothing useful: the command catches every failure of the lookup and falls back to the numeric collation id, which is not empty either, so it passed even if the changed call always threw. It compares against the collation the database really had, read before it is detached. The help of Connect-DbaInstance said to run statements through the connection context rather than a database object, without saying when. It now says for statements that do not depend on a particular database, and why the other kind still has to name one. (do Set-DbaTempDbConfig, Get-DbaDbDetachedFileInfo, Connect-DbaInstance)
Type of Change
Purpose
Step one of #10555: the mechanism, on its own.
The
QueryandInvokescript methods ofServerandDatabaseinxml/dbatools.Types.ps1xmldo not run on a private connection. The execution manager of an SMO database is the connection context of the parent server, which belongs to the caller, so they issue aUSEand never switch back:These four methods are reached from roughly 68 database-scoped
.Query()/.Invoke()call sites plus 45 two-argument$server.Query($sql, $db)calls across 28 files, so this one file is the cheapest place in the module to fix it.Approach
Each method remembers
ConnectionContext.CurrentDatabaseand puts it back in afinally, so a query that throws restores the context as well.The
Serverpair needs its own copy of that.Server.QueryandServer.Invokecall$this.Databases[$Database].ExecuteWithResults(...)directly and never go through theDatabasemethods, so fixingDatabase.Queryalone does not reach them. That is four edits, not two - the issue body assumed otherwise.Restoring, not copying.
ConnectionContext.Copy().GetDatabaseConnection($name)also works, and was the other candidate, but it is a different session:A copy cannot see the temp tables or
SEToptions of the caller and opens a connection per call, which would be a silent behaviour change for any command that builds session state and then queries through the wrapper. Restoring keeps the session and costs one round trip, and only when the database actually moved.The caller's database is restored, not master. A connection sitting in
msdbis returned tomsdb. Restoring to master would have passed every other test and still been wrong - and it matters, because the Agent commands move the context tomsdbrather thanmaster.The database name is escaped for the
USE, so a database containing]in its name is handled.Commands to test
Tests
tests\InModule.TypeExtensions.Tests.ps1, following the existingInModule.*naming for test files that are not the test of a single command. 10 tests onInstanceSingle:AllTablesstill returns every tablemsdbis returned tomsdb, not to masterAll 10 pass. Against
development7 of them fail; the 3 that pass are the correctness assertions, which are there to catch the fix breaking something rather than to prove the bug.Because this reaches every command that uses the wrappers, 15 further test files of wrapper-using commands were run on top:
Find-DbaSimilarTable,Get-DbaCpuRingBuffer,Get-DbaDatabase,Get-DbaDbFeatureUsage,Get-DbaDbFile,Get-DbaDbSnapshot,Get-DbaDbVirtualLogFile,Get-DbaHelpIndex,Get-DbaInstanceInstallDate,Get-DbaModule,Get-DbaSchemaChangeHistory,Install-DbaWhoIsActive,Invoke-DbaDbClone,New-DbaLinkedServer,Set-DbaDbFileGrowth. 104 tests, no failures.What this does not fix
Only the script methods. The other two sources in #10555 are untouched and still leak:
$db.ExecuteNonQuery(...)/$db.ExecuteWithResults(...)call sites, which are SMO's own methods and cannot be shadowedCreate()andDrop()of server-level objectsInvoke-DbaDbUpgrade(#10556) is in the first of those groups. Verified against a database forced to compatibility level 100 so the upgrade really ran - it went to 150 and the connection was still left in the upgraded database.🤖 Generated with Claude Code